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(54) Entertainment system, entertainment apparatus, recording medium, and program 



(57) An audio data processing means (200) regis- 
ters one of audio data extracted from a music CD (20) 
as one of sound sources for an entertainment apparatus 
(12). The audio data processing means (200) has an 
audio data extracting means (21 6) for sampling desired 



audio data from the audio data introduced via an optical 
disk drive (70) based on a control input entered from a 
manual controller (16), and an audio data registering 
means (226) for registering the sampled audio data as 
one of the sound sources. 
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Description 

BACKGROUND OF THE INVENTION 

Field of the Invention: 5 

[0001] The present invention relates to an entertain- 
ment system having at least one manual controller con- 
nected to an entertainment apparatus which executes 
various programs, for entering control requests from the 
user into the entertainment apparatus, an entertainment 
apparatus which executes various programs, a record- 
ing medium storing a program and data that are used 
by the entertainment system, and a program itself. 

Description of the Related Art: 

[0002] Some entertainment systems including enter- 
tainment apparatus such as video game machines dis- 
play video game images based on video game data 
stored in a recording medium such as a CD-ROM or the 
like on the display screen of a television receiver while 
allowing the user or game playerto play the video game 
with commands entered via a manual controller. 
[0003] In those entertainment systems, the entertain- 
ment apparatus and the manual controller are usually 
connected to each other by a serial interface. When a 
clock signal is supplied from the entertainment appara- 
tus to the manual controller, the manual controller sends 
key switch information based on the user's control en- 
tries in synchronism with the clock signal. 
[0004] Recently developed manual controllers incor- 
porate a vibration generating means for applying vibra- 
tions to the user based on a request from an external 
apparatus such as an entertainment apparatus, for ex- 
ample. While a video game is in progress, the vibration 
generating means applies various different kinds of vi- 
brations to the user in response to user's different con- 
trol entries. 

[0005] Almost all video games that can be played on 
the above entertainment system output BGM (Back- 
Ground Music) using a predetermined sound source. 
[0006] If audio data supplied from music compact 
discs (CDs) or via a network can be used as a BGM 
sound source for video games, then a wide variety of 
existing music data is made available for use in video 
games, providing an additional element of interest for 
video games. 

SUMMARY OF THE INVENTION 

[0007] Embodiments of the present invention seek to 
provide an entertainment system, an entertainment ap- 
paratus, a recording medium, and a program which al- 
low audio data suppliedf rom music CDs or via a network 
to be used as a BGM sound source forthe entertainment 
apparatus, thus providing an additional element of inter- 
est for video games. 



[0008] According to a first aspect of the present inven- 
tion there is provided an entertainment system compris- 
ing an entertainment apparatus for executing various 
programs, at least one manual controller for entering 
control requests from the user into the entertainment ap- 
paratus, audio output means for selecting and output* 
ting one of a plurality of sound sources based on a sup- 
plied sound output command, a display unit for display- 
ing images outputted from the entertainment apparatus, 
and audio data processing means for registering ex- 
tracted one of audio data introduced from an external 
source as one of the sound sources for the entertain- 
ment apparatus. 

[0009] According to a second aspect of the present 
invention there is provided an entertainment apparatus 
for connection to a manual controller for outputting at 
least a control request from the user, a display unit for 
displaying images, and audio output means for selecting 
and outputting one of a plurality of sound sources based 
on a supplied sound output command, comprising audio 
data processing means for registering extracted one of 
audio data introduced from an external source as one 
of the sound sources. 

[0010] According to a third aspect of the present in- 
vention, there is also provided a recording medium stor- 
ing a program and data for use in an entertainment sys- 
tem having an entertainment apparatus for executing 
various programs, at least one manual controller for en- 
tering control requests from the user into the entertain- 
ment apparatus, audio output means for selecting and 
outputting one of a plurality of sound sources based on 
a supplied sound output command, and a display unit 
for displaying images outputted from the entertainment 
apparatus, the program comprising the step of register- 
ing extracted one of audio data introduced from an ex- 
ternal source as one of the sound sources for the enter- 
tainment apparatus. 

[001 1] According to a fourth aspect of the present in- 
vention, there is also provided a program readable and 
executable by a computer, for use in an entertainment 
system having an entertainment apparatus for execut- 
ing various programs, at least one manual controller for 
entering control requests from the user into the enter- 
tainment apparatus, audio output means for selecting 
and outputting one of a plurality of sound sources based 
on a supplied sound output command, and a display unit 
for displaying images outputted from the entertainment 
apparatus, the program comprising the step of register- 
ing extracted one of audio data introduced from an ex- 
ternal source as one of the sound sources for the enter- 
tainment apparatus. 

[001 2] With the above arrangement, audio data intro- 
duced from a music CD or via a network can be used 
as a sound source for the entertainment apparatus, pro- 
viding an additional element of musical interest to video 
games that are played back on the entertainment appa- 
ratus. 

[001 3] The audio data processing means or step may 
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comprise audio data extracting means for, or the step 
of, extracting audio data from the audio data introduced 
from the external source based on a control input en- 
tered from the manual controller, and audio data regis- 
tering means for, or the step of, registering the extracted 
audio data as one of the sound sources. 
[0014] The audio data processing means or step may 
comprise trimming means for, or the step of, trimming 
an excessive portion off the extracted audio data. Since 
unwanted data can be cut off from the extracted audio 
data, only impressive audio data can be used as a sound 
source. 

[001 5] The audio data processing means or step may 
comprise effect applying means for, or the step of, ap- 
plying an effect to the extracted audio data. Thus, even 
one form of audio data can be modified into different 
forms of audio data by applying various effects, resulting 
in a variety of sound sources available. 
[001 6] The audio data processing means or step may 
comprise audio data re-extracting means for, or the step 
of, re- extracting audio data from the extracted audio da- 
ta. Consequently, the user can extract only favorite por- 
tions from the audio data to which various effects have 
been applied or the audio data from which unwanted 
portions have been removed. Therefore, the quality of 
audio data for use as sound sources can be increased, 
and the process of extracting audio data is easy. 
[0017] The audio data re-extracting means or step 
may comprise selecting playback means for reproduc- 
ing the extracted audio data according to a playback at- 
tribute selected according to a control input entered from 
the manual controller. Therefore, the audio data can be 
varied in various playback modes such as the reverse 
playback mode and the skipping playback mode, result- 
ing in a variety of sound sources available. 
[001 8] Further particular and preferred aspects of the 
present invention are set out in the accompanying inde- 
pendent and dependent claims. Features of the depend- 
ent claims may be combined with features of the inde- 
pendent claims as appropriate, and in combinations oth- 
er than those explicitly set out in the claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 



mechanisms disposed respectively in left and right 
grips thereof; 

FIG. 6 is a block diagram of a circuit arrangement 
of an entertainment apparatus; 
5 FIG. 7 is a block diagram of the manual controller; 
FIG. 8 is a block diagram of components for carrying 
out bidirectional serial communications between 
the manual controller and the entertainment appa- 
ratus; 

10 FIG. 9 is a functional block diagram of a audio data 
processing means including a sampling means; 
FIG. 10 is a functional block diagram of a re-sam- 
pling means in the audio data processing means; 
FIG. 11 is a flowchart of a processing sequence of 

is the audio data processing means; 

FIGS. 1 2 and 1 3 are a flowchart of a processing se- 
quence of the sampling means; 
FIG. 14 is a flowchart of a processing sequence of 
a sound sampling means; 

20 FIG. 15 is a flowchart of a processing sequence of 
a slot selecting means; 

FIG. 16 is a flowchart of a processing sequence of 

a first command selecting means; 

FIG. 17 is a flowchart of a processing sequence of 
25 a waveform editing means; 

FIGS. 1 8 and 1 9 are a flowchart of a processing se- 
quence of the re-sampling means; 

FIG. 20 is a flowchart of a processing sequence of 

a slot assigning means; 
30 FIG. 21 is a flowchart of a processing sequence of 

a playback processing means; 

FIG. 22 is a flowchart of a processing sequence of 

a sound re-sampling means; 

FIG. 23 is a flowchart of a processing sequence of 
35 a second command selecting means; 

FIG. 24 is a view showing a displayed sampling 

view; 

FIG . 25 is a view showing a displayed waveform ed- 
iting view; and 

40 FIG. 26 is a view showing a displayed re-sampling 
view. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 



[001 9] The present invention will be described further, 
by way of example only, with reference to preferred em- 
bodiments thereof as illustrated in the accompanying 
drawings, in which: 

FIG. 1 is a perspective view of an entertainment 
system according to an embodiment of the present 
invention; 

FIG. 2 is a perspective view of a manual controller; 
FIG. 3 is a plan view of the manual controller; 
FIG. 4 is a perspective view showing the manner in 
which the manual controller is used; 
FIG. 5 is a bottom view, partly broken away, of the 
manual controller, showing vibration imparting 



45 [0020] An entertainment system and an entertain- 
ment apparatus according to embodiments of the 
present invention as applied to a video game apparatus, 
and a recording medium and a program according to 
embodiments of the present invention as applied to a 
so recording medium which stores a program and data to 
be executed by the video game apparatus and a pro- 
gram to be executed by the video game apparatus will 
be described below with reference to FIGS. 1 through 
26. 

55 [0021] As shown in FIG. 1 , an entertainment system 
10 basically comprises an entertainment apparatus 12 
for executing various programs, a memory card 14 de- 
tachably connected to the entertainment apparatus 12, 
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a manual controller 1 6 detachably connected to the en- 
tertainment apparatus 12 by a connector 62, and a dis- 
play monitor 18 such as a television receiver which is 
supplied with video and audio output signals from the 
entertainment apparatus 12. 

[0022] The entertainment apparatus 1 2 reads a pro- 
gram recorded in a mass storage medium such as an 
optical disk 20 such as a CD-ROM or the like, and exe- 
cutes a game, for example, based on the program de- 
pending on commands supplied from the user, e.g., the 
game player, via the manual controller 16. The execu- 
tion of the game mainly represents controlling the 
progress of the game by controlling the display of imag- 
es and the generation of sounds on the display monitor 
1 8 based on manual input actions entered from the man- 
ual controller 1 6 via the connector 62. 
[0023] The entertainment apparatus 12 has a sub- 
stantially flat casing in the shape of a rectangular paral- 
lelepiped which houses a disk loading unit 22 disposed 
centrally for loading an optical disk 20 for supplying an 
application program and data for a video game or the 
like. The casing supports a reset switch 24 for resetting 
a program which is being presently executed, a disk 
control switch 26 for controlling the loading of the optical 
disk 20, a power supply switch 28, and two slots 30, 32. 
[0024] The entertainment apparatus 12 may be sup- 
plied with the application program via a communication 
link, rather than being supplied from the optical disk 20 
as the recording medium. 

[0025] The slots 30, 32 have respective upper slot 
units 30B, 32 B and respective lower slots units 30 A, 
32A. Two manual controllers 16 may be connected re- 
spectively to the lower slots units 30A, 32 A, and memory 
cards 14 or portable information terminals (not shown) 
having the function of the memory card 14 for storing 
flags indicative of interim game data may be connected 
respectively to the upper slots units 30B, 32B. The slots 
30, 32 (the upper slot units 30B, 32B and the lower slots 
units 30A, 32A) are asymmetrically shaped to prevent 
the connectors 62 and the memory cards 1 4 from being 
inserted in the wrong direction. 

[0026] As shown in FIGS. 2 and 3, the manual con- 
troller 16 has first and second control pads 34, 36, an L 
(Left) button 38L, an R (Right) button 38R, a start button 
40, and a selection button 42. The manual controller 1 6 
also has joysticks 44, 46 for inputting analog control ac- 
tions, a mode selection switch 48 for selecting control 
modes of the joysticks 44, 46, and a mode indicator 50 
for indicating a selected control mode. The mode indi- 
cator 50 comprises a light-emitting element such as a 
light-emitting diode or the like. 

[0027] As shown in FIG. 2, the manual controller 16 
has a housing 104 comprising an upper member 100 
and a lower member 1 02 which are mated and joined to 
each other by fasteners such as screws. 
[0028] As shown in FIGS. 2 and 3, a pair of left and 
right grips 1 06, 1 08 projects from one side of respective 
opposite ends of the housing 1 04. The left and right grips 



106, 108 are shaped so as to be gripped by the palms 
of left and right hands of the user or game player when 
the manual controller 16 is connected to the entertain- 
ment apparatus 1 2 and information retrieval is carried 

5 out or the game is played thereby, for example. 

[0029] As shown in FIG. 3, the left and right grips 1 06, 
1 08 are progressively spaced away from each other to- 
ward their distal ends. To allow the game player to grip 
the left and right grips 106, 108 comfortably for a long 

10 period of time, the left and right grips 106, 108 are ta- 
pered from their joint with the housing 104 toward their 
distal ends, and have arcuate outer peripheral surfaces 
and arcuate distal end surfaces. 

[0030] As shown in FIGS. 2 and 3, the first control pad 
15 34 is disposed on one end of the housing 1 04 and com- 
prises a first pressable control member (up button) 
1 1 0a, a second pressable control member (right button) 
110b, a third pressable control member (down button) 
110c, and a fourth pressable control member (left but- 
20 ton) 110d. The first through fourth pressable control 
members 110a, 110b, 110c, 110d project on an upper 
surface of the housing 104 and are arranged in a criss- 
cross pattern. 

[0031] The first control pad 34 includes switch ele- 

25 ments as signal input elements associated respectively 
with the first through fourth pressable control members 
110a, 110b, 110c, 110d. The first control pad 34 func- 
tions as a directional controller for controlling the direc- 
tion of movement of a displayed game character, for ex- 

30 ample. When the game player selectively presses the 
first through fourth pressable control members 110a, 
110b, 110c, 110d to turn on or off the switch elements 
associated respectively with the first through fourth 
pressable control members 110a, 110b, 110c, 11 Od, the 

35 displayed game character moves in the direction corre- 
sponding to the pressed one of the first through fourth 
pressable control members 110a, 110b, 110c, 110d. 
[0032] As shown in FIGS. 2 and 3, the second control 
pad 36 is disposed on the other end of the housing 104 

40 and comprises a first pressable control member (A but- 
ton) 112a, a second pressable control member (□ but- 
ton) 112b, a third pressable control member (x button) 
1 1 2c, and a fourth pressable control member (O button) 
11 2d. The first through fourth pressable control mem- 

45 bers 112a, 112b, 112c, 11 2d project on the upper sur- 
face of the housing 1 04 and are arranged in a crisscross 
pattern. 

[0033] The first through fourth pressable control 
members 1 1 2a, 1 1 2b, 1 1 2c, 1 1 2d are constructed as in- 

50 dependent members, and associated with respective 
switch elements disposed in the second control pad 36. 
[0034] The second control pad 36 serves as a function 
setting/performing unit for setting functions for a dis- 
played game character assigned to the pressable con- 

55 trol members 112a - 11 2d or performing functions of a 
displayed game character when the switch elements as- 
sociated with the pressable control members 112a - 
11 2d are turned on. 
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[0035] The L button 38L and the R button 38R are dis- 
posed on a side of the housing 104 remote from the left 
and right grips 106, 108 and positioned respectively at 
the opposite ends of the housing 104. Asshown in FIGS. 
2 and 4, the L button 38L has a first left pressable control 
member (L1 button) 114a and a second left pressable 
control member (L2 button) 1 1 4b, and the R button 38R 
has a first right pressable control member (R1 button) 
116a and second right pressable control member (R2 
button) 116b, respectively. 

[0036] The L button 38L and the R button 38R have 
respective switch elements associated respectively with 
the pressable control members (the L1 button 1 14a, the 
L2 button 114b, the R1 button 116a, and the R2 button 
116b). 

[0037] The L button 38L and the R button 38R serve 
as respective function setting/performing units for set- 
ting functions for a displayed game character assigned 
to the pressable control members 1 1 4a, 1 1 4b and 1 1 6a, 
1 1 6b or performing functions of a displayed game char- 
acter when the switch elements associated with the 
pressable control members 114a, 114b and 116a, 116b 
are turned on. 

[0038] As shown in FIGS. 2 and 3, the manual con- 
troller 1 6 also has first and second analog control pads 
118, 120 disposed respectively at confronting corners 
defined between the housing 1 04 and the proximal ends 
of the left and right grips 106, 108 which are joined to 
the housing 104. 

[0039] The first and second analog control pads 1 1 8, 
120 have the respective joysticks 44, 46 which can be 
tilted in all directions (360°) about control shafts thereof, 
and respective signal input elements such as variable 
resistors or the like which are operable by the respective 
joysticks 44, 46. Specifically, the control shafts of the left 
and right joysticks 44, 46 are normally urged to return 
to their neutral positions by biasing members. The left 
and the right joysticks 44, 46 can be freely tilted in all 
directions (360°) about the axes of the control shafts. 
[0040] The first and second analog control pads 118, 
1 20 can move a displayed game characterwhile rotating 
the same or while changing its speed, and can make an 
analog-like action such as to change the form of a dis- 
played character, when the game player manipulates 
the joysticks 44, 46. Therefore, the first and second an- 
alog control pads 11 8, 1 20 are used as a control unit for 
entering command signals for a displayed character to 
perform the above movement or action. 
[0041 ] When the mode selection switch 48 is pressed, 
it can select a control mode for allowing a command sig- 
nal to be inputted from the first and second analog con- 
trol pads 118, 120 or a control mode for inhibiting a com- 
mand signal from being inputted from the first and sec- 
ond analog control pads 118, 120. 
[0042] When the mode selection switch 48 is pressed, 
the functions of the first through fourth pressable control 
members 112a, 112b, 112c, 11 2d of the second control 
pad 36, and the functions of the pressable control mem- 



bers 1 1 4a, 1 1 4b and 1 1 6a, 1 1 6b of the L button 38L and 
the R button 38R are changed depending on the control 
mode selected by the pressed mode selection switch 
48. Depending on the control mode selected by the 
5 mode selection switch 48, the mode indicator 50 flickers 
and changes its indication light. 

[0043] As shown in FIG. 4, the left and right grips 1 06, 
1 08 projecting from the housing 1 04 are gripped respec- 
tively by the palms of the hands of the game player. The 
10 housing 1 04 is not required to be supported by fingers, 
and the manual controller 16 can be held by the hands 
while at least six out of the ten fingers of the hands can 
freely be moved. 

[0044] As shown in FIG. 4, when the first and second 

15 grips 1 06, 1 08 are gripped respectively by the palms of 
the hands of the game player, the thumbs Lf 1 , Rf 1 of the 
left and right hands can extend over the joysticks 44, 46 
of the first and second analog control pads 1 1 8, 1 20, the 
first through fourth pressable control members 110a - 

20 11 od of the first control pad 34, and the first through 
fourth pressable control members 112a - 11 2d of the 
second control pad 36, and can selectively press the joy- 
sticks 44, 46, the pressable control members 110a - 
110d, and the pressable control members 112a - 11 2d. 

25 [0045] Since the joysticks 44, 46 of the first and sec- 
ond analog control pads 1 1 8, 1 20 are positioned in con- 
fronting relation to the proximal ends of the left and right 
grips 106, 108 which are joined to the housing 104, 
when the left and right grips 1 06, 1 08 are gripped by the 

30 left and right hands, the joysticks 44, 46 are positioned 
most closely to the thumbs Lf1 , Rf1 , respectively. There- 
fore, the joysticks 44, 46 can easily be manipulated by 
the thumbs Lf 1 , Rf1. 

[0046] As shown in FIG. 4, when the left and right 
35 grips 1 06, 1 08 are gripped respectively by the palms of 
the hands of the game player, the index fingers Lf2, Rf2 
and middle fingers Lf3, Rf3 of the left and right hands 
can extend over positions where they can selectively 
press the L1 button 1 1 4a, L2 button 1 1 4b of the L button 
40 38L and R1 button 1 1 6a, R2 button 1 1 6b of the R button 
38R. 

[0047] As shown in FIG. 5, the manual controller 16 
has a pair of vibration imparting mechanisms 128L, 
128R for imparting vibrations to the user in order for the 

45 user to be able to play a highly realistic game. 

[0048] As shown in FIG. 5, the left and right vibration 
imparting mechanisms 128L, 128R are positioned near 
the proximal ends of the left and right grips 1 06, 1 08 that 
are held by the hands and fingers when the manual con- 

so troller 1 6 is gripped by the user. 

[0049] Since the both vibration imparting mecha- 
nisms 128L, 128R have basically the same structure ex- 
cept their vibration characteristics, only the right vibra- 
tion imparting mechanism 1 28R will be described forthe 

55 purpose of brevity. 

[0050] The vibration imparting mechanisms 128R 
comprises a motor 1 30R energizable by a vibration gen- 
erating command supplied from the entertainment ap- 
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paratus 12, and an eccentric member 134R mounted 
eccentrically on the drive shaft of the motor 130R. 
[0051] The eccentric member 134R comprises a 
weight in the form of a heavy metal member having a 
semicircular cross -sectional shape. The weight has an 
off-center hole defined therein in which the drive shaft 
of the motor 130R is fitted. 

[0052] According to the vibration imparting mecha- 
nisms 128L, 128R as constructed above, when the mo- 
tors 130L, 130R are energized, the drive shafts thereof 
rotate to cause the eccentric members 134L, 134R to 
rotate in an eccentric motion for thereby generating vi- 
brations, which are imparted to the left grip 1 06 and the 
right grip 108. Then, the vibrations of the left grip 106 
and the right grip 1 08 are applied to the hands and fin- 
gers of the user. 

[0053] Next, the vibration characteristics of the vibra- 
tion imparting mechanisms 128L, 128R disposed in the 
left grip 1 06 and the right grip 1 08 respectively will be 
described hereinbelow. 

[0054] The vibration imparting mechanisms 128L, 
128R have the different vibration characteristics. 
[0055] For example, the motor 130L of the left vibra- 
tion imparting mechanism 128L is biggerthan the motor 
130R of the right vibration mechanism 128R. The rota- 
tional speed of the motor 130L varies according to a vi- 
bration value included in a vibration generating com- 
mand transmitted from the entertainment apparatus 12. 
That is, vibrations having different frequencies can be 
generated depending on the vibration value. In the 
present embodiment, the vibration frequency of the mo- 
tor 130L varies in proportion to the vibration value. 
[0056] In contrast to the motor 130L of the left vibra- 
tion mechanism 1 28L, the vibration frequency of the mo- 
tor 130R of the right vibration mechanism 128R does 
not vary according to the vibration value included in the 
vibration generating command. The motor 130R of the 
right vibration mechanism 128R is simply either ener- 
gized or de-energized according to the vibration value. 
If the vibration value (logic value) is "I", the motor 130R 
of the right vibration mechanism 128R is energized. If 
the vibration value is "0", the motor 130R of the right 
vibration mechanism 128R is de-energized. When the 
motor 130R of the right vibration mechanism 128R is 
energized, it rotates at a constant speed to generate vi- 
brations at a constant frequency. 

[0057] In order to energize the motors 130L, 130Rto 
vibrate the manual controller 1 6 in its entirety, a bidirec- 
tional communication function needs to be provided be- 
tween the manual controller 16 and the entertainment 
apparatus 1 2. This bidirectional communication function 
will be described later on. 

[0058] Now, circuit arrangements of the entertain- 
ment apparatus 12 and the manual controller 16 will be 
described below with reference to FIGS. 6 through 8. 
[0059] As shown in FIG. 8, the entertainment appara- 
tus 1 2 generally comprises a control system 60, a graph- 
ic generating system 64 connected to the control system 



60 via a system bus 61 , a sound generating system 66 
connected to the control system 60 via the system bus 
61 , and an optical disk control system 68 connected to 
the control system 60 via the system bus 61 . A commu- 
5 nication controller 58 for controlling data to be inputted 
to and outputted from the manual controller 16 and the 
memory card 14 is also connected to the control system 
60 via the system bus 61 . 

[0060] The manual controller 1 6 supplies commands 

10 (including control data) from the user via a communica- 
tion controller 150 (see FIG. 7) of the manual controller 
1 6 and the communication controller 58 to the entertain- 
ment apparatus 12. The optical disk control system 68 
includes an optical disk drive 70 in which the optical disk 

15 20, which may comprise a CD-ROM or the like as a spe- 
cific example of a recording medium according to an em- 
bodiment of the present invention. 
[0061] The control system 60 controls motions of 
characters displayed on the monitor 18 based on a pro- 

20 gram and data read from the optical disk 20 and com- 
mands supplied from the manual controller 16. 
[0062] The control system 60 includes a central 
processing unit (CPU) 72, a peripheral device controller 
74 for controlling interrupts and direct memory access 

25 (DMA) data transfer, a main memory 76 comprising a 
random-access memory (RAM), and a read-only mem- 
ory (ROM) 78 which stores various programs such as 
an operating system for managing the graphic generat- 
ing system 64, the sound generating system 66, etc. The 

30 main memory 76 can store at least a game program that 
is supplied from the optical disk 20 and executed by the 
central processing unit 72. 

[0063] The CPU 72 controls the entertainment appa- 
ratus 1 2 in its entirety by executing the operating system 
35 stored in the ROM 78. The CPU 72 comprises a 32-bit 
RISC-CPU, for example. 

[0064] When the entertainment apparatus 12 is 
turned on, the CPU 72 executes the operating system 
stored in the ROM 78 to start controlling the graphic gen- 

40 erating system 64, the sound generating system 66, etc. 
[0065] When the operating system is executed, the 
CPU 72 initializes the entertainment apparatus 12 in its 
entirety for confirming its operation, and thereafter con- 
trols the optical disc control system 68 to execute an 

45 application program such as a game program recorded 
in the optical disk 20. 

[0066] As the application program such as a game 
program is executed, the CPU 72 controls the graphic 
generating system 64, the sound generating system 66, 

50 etc. depending on commands entered from the user for 
thereby controlling the display of images and the gen- 
eration of music sounds and sound effects. 
[0067] The graphic generating system 64 comprises 
a geometry transfer engine (GTE) 80 for performing co- 

55 ordinate transformations and other processing, a graph- 
ic processing unit (GPU) 82 for rendering image data 
according to instructions from the CPU 72, a frame buff- 
er 84 for storing image data rendered by the GPU 82, 
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and an image decoder 86 for decoding image data com- 
pressed and encoded by an orthogonal transform such 
as a discrete cosine transform. 

[0068] The GTE 80 has a parallel arithmetic mecha- 
nism for performing a plurality of arithmetic operations 
parallel to each other, and can perform coordinate trans- 
formations and light source calculations, and calculate 
matrixes or vectors at a high speed in response to a re- 
quest from the CPU 72. 

[0069] Specifically, the GTE 80 can calculate the co- 
ordinates of a maximum of 1 .5 million polygons per sec- 
ond for a flat shading process to plot one triangular pol- 
ygon with one color, for example. With the GTE 80, the 
entertainment apparatus 1 2 is able to reduce the burden 
on the CPU 72 and perform high-speed coordinate cal- 
culations. 

[0070] According to an image generating instruction 
from the CPU 72, the GPU 82 generates and stores the 
data of a polygon or the like in the frame buffer 84. The 
GPU 82 is capable of generating and storing a maximum 
of 360 thousand polygons per second. 
[0071] The frame buffer 84 comprises a dual-port 
RAM, and is capable of simultaneously storing image 
data generated by the GPU 82 or image data transferred 
from the main memory 76, and reading image data for 
display. The frame buffer 84 has a storage capacity of 
1 Mbytes, for example, and is handled as a 16-bit matrix 
made up of a horizontal row of 1 024 pixels and a vertical 
column of 512 pixels. 

[0072] The frame buffer 84 has a display area for stor- 
ing image data to be outputted as video output data, a 
CLUT (color look-up table) area for storing a color look- 
up table which will be referred to by the GPU 82 when 
it renders a polygon or the like, and a texture area for 
storing texture data to be subjected to coordinate trans- 
formations when a polygon is generated and mapped 
onto a polygon generated by the GPU 82. The CLUT 
area and the texture area are dynamically varied as the 
display area is varied. 

[0073] The GPU 82 can perform, in addition to the flat 
shading process, a Gouraud shading process for deter- 
mining colors in polygons by interpolating intensities 
from the vertices of the polygons, and a texture mapping 
process for mapping textures stored in the texture area 
onto polygons. For performing the Gouraud shading 
process or texture mapping process, the GTE 80 can 
perform coordinate calculations for a maximum of about 
500,000 polygons per second. 

[0074] The image decoder 86 is controlled by the CPU 
72 to decode image data of a still or moving image 
stored in the main memory 76, and store the decoded 
image into the main memory 76. 

[0075] Image data reproduced by the image decoder 
86 is transferred to the frame buffer 84 by the GPU 82, 
and can be used as a background for an image plotted 
by the GPU 82. 

[0076] The sound generating system 66 comprises a 
sound processing unit (SPU) 88 for generating music 



sounds, sound effects, etc. based on instructions from 
the CPU 72, and a sound buffer 90 for storing music 
sounds, sound effects, etc. generated by the SPU 88. 
Audio signals representing music sounds, sound ef- 
5 fects, etc. generated by the SPU 88 are supplied to au- 
dio terminals of the monitor 18. The monitor 18 has a 
speaker 92 which radiates music sounds, sound effects, 
etc. generated by the SPU 88 based on the supplied 
audio signals. 

10 [0077] The SPU 88 has an ADPCM (adaptive differ- 
ential PCM) function for reproducing 16-bit audio data 
which has been encoded as 4-bit differential audio data 
by ADPCM, a reproducing function for reproducing 
waveform data stored in the sound buffer 90 to generate 

15 sound effects, etc., and a modulating function for mod- 
ulating and reproducing the waveform data stored in the 
sound buffer 90. 

[0078] The sound system 66 with these functions can 
be used as a sampling sound source which generates 
20 music sounds, sound effects, etc. based on the wave- 
form data stored in the sound buffer 90 according to in- 
structions from the CPU 72. 

[0079] The optical disk control system 68 comprises 
an optical disk drive 70 for reproducing application pro- 

25 grams and data recorded on the optical disk 20, a de- 
coder 94 for decoding programs and data that are re- 
corded with an error correcting code added thereto, and 
a buffer 96 for temporarily storing data read from the 
optical disk drive 70 so as to allow the data from the 

30 optical disk 20 to be read at a high speed. An auxiliary 
CPU 98 is connected to the decoder 94. 
[0080] Audio data recorded on the optical disk 20 
which is read by the optical disk drive 70 includes PCM 
data converted from audio signals, in addition to the AD- 

35 PCM data. 

[0081] The ADPCM data, which is recorded as 4-bit 
differential data of 1 6-bit digital data, is decoded by the 
decoder 94, supplied to the SPU 88, converted thereby 
into audio signals, and applied to drive the speaker 92. 

40 [0082] The PCM data, which is recorded as 1 6-bit dig- 
ital data, is decoded by the decoder 94 and then applied 
to drive the speaker 92. 

[0083] As shown in FIG. 7, the manual controller 16 
comprises a communication controller 150, a CPU 152, 

45 a program memory 154, a working RAM 156, a digital 
input block 158, an analog input block 1 60, a left motor 
driver 1 70L for energizing the left motor 1 30L, and a right 
motor driver 170R for energizing the right motor 130R. 
These components of the communication controller 16 

50 are connected to a bus 1 62. 

[0084] The digital input block 1 58 functions as a man- 
ual input controller for the pressable control members 
1 1 0a - 1 1 0d of the first control pad 34 and the pressable 
control members 1 1 2a - 1 1 2d of the second control pad 

55 36. The analog input block 160 functions as a manual 
input controller for the left and right joysticks 44, 46. The 
digital input block 158 and the analog input block 160 
allow the user to enter various items of information into 
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the manual controller 16. 

[0085] The communication controller 150 has a func- 
tion to effect serial communications with an external de- 
vice. The communication controller 150 is electrically 
connectable to the communication controller 58 (see 
FIG. 6) of the entertainment apparatus 12, for example, 
for data communications with the entertainment appa- 
ratus 12. 

[0086] As shown in FIG. 8, the bidirectional commu- 
nication function between the entertainment apparatus 
1 2 and the manual controller 1 6 can be performed when 
the connector 62 capable of performing bidirectional se- 
rial communications with the manual controller 16 is 
connected to the entertainment apparatus 12. 
[0087] A system in the manual controller 16 for per- 
forming the bidirectional communication function com- 
prises a serial I/O interface SIO for performing serial 
communication with the entertainment apparatus 12, a 
parallel I/O interface PIO for entering control data from 
a plurality of control buttons, a one-chip microcomputer 
comprising a CPU, a RAM, and a ROM, and a pair of 
motor drivers 170R, 170L for energizing the motors 
130R, 130L of the vibration imparting mechanisms 
128R, 128L. Each of the motors 130R, 130L is ener- 
gized by a voltage and a current supplied from the motor 
drivers 170R, 170L. 

[0088] A system in the entertainment apparatus 1 2 for 
performing the bidirectional communication function 
comprises a serial I/O interface SIO forperforming serial 
communication with the manual controller 1 6 . When the 
connector 62 is connected to the serial I/O interface SIO 
of the entertainment apparatus 12, the serial I/O inter- 
face SIO of the entertainment apparatus 12 is connect- 
ed to the serial I/O interface SIO of the manual controller 
1 6 via the connector 62 forperforming bidirectional com- 
munications between the entertainment apparatus 12 
and the manual controller 16. Other detailed structure 
of the entertainment apparatus 1 2 are omitted from il- 
lustration in FIG. 8. 

[0089] Signal and control lines for bidirectional serial 
communications include a data transfer signal lineTXD 
(Transmit X' for Data) for sending data from the enter- 
tainment apparatus 1 2 to the manual controller 1 6, a da- 
ta transfer signal line RXD (Received X* for Data) for 
sending data from the manual controller 1 6 to the enter- 
tainment apparatus 1 2, a serial synchronous clock sig- 
nal line SCK (Serial Clock) for extracting data from the 
data transfer signal lines TXD, RXD, a control line DTR 
(Data Terminal Ready) for establishing and cutting off 
communication with the manual controller 16 as a ter- 
minal, and a flow control line DSR (Data Set Ready) for 
transferring a large amount of data. 
[0090] As shown in FIG. 8, the signal and control lines 
for bidirectional serial communication are accommodat- 
ed in a cable. This cable further includes a power line 
1 72 extending from a power supply in the entertainment 
apparatus 12 and connected to the motor drivers 1 70R, 
170L in the manual controller 16 for supply electric en- 



ergy to energize the motors 130R, 130L. 
[0091] A process of bidirectional serial communica- 
tion between the entertainment apparatus 12 and the 
manual controller 16 will be described below. In order 
s for the entertainment apparatus 12 to communicate with 
the manual controller 1 6 to read control data from the 
digital input block 158 and the analog input block 160, 
the entertainment apparatus 12 first outputs selection 
data to the control line DTR. As a result, the manual con- 
io troller 16 confirms that it is selected by the control line 
DTR, and then waits for a signal from the signal line 
TXD. Then, the entertainment apparatus 12 outputs an 
identification code indicative of the manual controller 16 
to the data transfer signal line TXD. The manual con- 
's troller 1 6 receives the identification code from the signal 
line TXD. 

[0092] When the manual controller 1 6 recognizes the 
identification code, the manual controller 1 6 starts com- 
municating with the entertainment apparatus 12. The 

20 entertainment apparatus 12 sends control data via the 
data transfer signal line TXD to the manual controller 
1 6, which sends control data from the digital input block 
1 58 and the analog input block 1 60 via the data transfer 
signal line RXD to the entertainment apparatus 12. In 

25 this manner, the entertainment apparatus 12 and the 
manual controller 16 perform bidirectional serial com- 
munications. The bidirectional serial communications 
will be finished when the entertainment apparatus 12 
outputs selection stop data via the control line DTR. 

30 [0093] With the bidirectional serial communication 
function, the manual controller 1 6 can send mainly con- 
trol data from the digital input block 158 and the analog 
input block 160 to the entertainment apparatus 12, and 
the entertainment apparatus 1 2 can send vibration gen- 

35 erating commands for energizing the motors 130R, 
1 30L of the vibration imparting mechanisms 1 28R, 1 28L 
via the data transfer signal line TXD to the manual con- 
troller 16. 

[0094] The vibration generating commands for ener- 

40 gizing the motors 1 30R, 1 30L include those which have 
been established in advance in the optical disk 20 set 
in the entertainment apparatus 12 and those which are 
newly generated in the entertainment apparatus 12. 
[0095] A characteristic function of the entertainment 

45 system 1 0 according to the present embodiment will be 
described below with reference to FIGS. 9 through 26. 
[0096] According to the characteristic function, audio 
data extracted from audio data introduced from an ex- 
ternal source is registered as one of sound sources for 

so the entertainment apparatus 12. 

[0097] Specifically, audio data from the optical disk 20 
loaded in the optical disk drive 70 of the entertainment 
apparatus 12 or a network connected to the entertain- 
ment apparatus 1 2 is introduced, and desired audio data 

55 is extracted from the introduced audio data and used as 
a sound source for the entertainment apparatus 12. In 
this embodiment, audio data is introduced from a music 
CD (Compact Disc) loaded in the optical disk drive 70. 
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[0098] One example of software for performing the 
above characteristic function will be described below 
with reference to FIGS. 9 through 26. As shown in FIG. 
9, the software comprises a audio data processing 
means 200. 

[0099] The audio data processing means 200 can be 
supplied to the entertainment system 1 0 from a random- 
ly accessible recording medium such as a CD-ROM , the 
memory card 14, or a network. It is assumed in the 
present embodiment that the audio data processing 
means 200 is read from the optical disk 20 such as a 
CD-ROM into the entertainment apparatus 12. 
[0100] The audio data processing means 200 is 
downloaded in advance from the optical disk 20 played 
back by the entertainment apparatus 12 into the main 
memory 76 in the control system 60 thereof according 
to a predetermined process, and executed by the CPU 
72 of the control system 60. 

[01 01 ] As shown in FIG. 9, the audio data processing 
means 200 has a CD checking means 202 for checking 
whether the optical disk 20 loaded in the optical disk 
drive 70 is a music CD or not, and a sampling means 
204 for sampling any desired music data from audio da- 
ta introduced when a music CD is played back. 
[0102] The sampling means 204 comprises a CD con- 
trolling means 210 for controlling the optical disk drive 
70 to play, stop, access tracks (forward and backward) 
of, and otherwise operate a music CD loaded in the op- 
tical disk drive 70 in response to control inputs from the 
manual controller 16, an effect applying means 212 for 
applying various effects to reproduced music data from 
the music CD in response to control inputs from the 
manual controller 16, a parameter changing means 214 
for changing parameters including an effect level, a 
sound volume, etc. of audio data in response to control 
inputs from the manual controller 1 6, a sound sampling 
means 216 for sampling audio data of a portion of the 
reproduced audio data which is determined by a control 
input from the manual controller 16 and registering the 
sampled audio data in a predetermined recording time 
slot, a slot selecting means 21 8 for selecting a recording 
slot to register sampled audio data therein, and a first 
command selecting means 220 for selecting various 
commands after music data is sampled. 
[01 03] The effect applying means 212 controls the op- 
tical disk drive 70 to appiy a necessary effect to repro- 
duced audio data. 

[0104] The first command selecting means 220 com- 
prises a waveform editing means 222 for removing ex- 
cessive data from sampled audio data, a re-sampling 
means 224 for re-sampling desired audio data from 
sampled audio data, and a sampled data registering 
means 226 for registering sampled audio data in a audio 
data file in the sound buffer 90. 

[01 05] A number of audio data registered in the audio 
data file serve as sound source data for use by the en- 
tertainment apparatus 12. When sampled audio data is 
registered in the audio data file by the sampled data reg- 



istering means 226, the sampled audio data is available 
as a sound source for the entertainment apparatus 12. 
[01 06] The sound sampling means 21 6 uses a record- 
ing slot file 228 which has six records corresponding to 
5 respective six recording slots, for example, that are han- 
dled by the sound sampling means 21 6. Each of the six 
records of the recording slot file 228 stores sampled au- 
dio data. 

[0107] As shown in FIG. 10, the re-sampling means 

10 224 comprises a slot assigning means 240 for selecting 
a recording slot to be assigned to a playback time slot 
(playback slot) for re-sampling, a playback processing 
means 242 for reproducing audio data assigned to play- 
back slots, a slot selecting means 244 for selecting a 

?5 playback slot to be played back, an effect applying 
means 246 for applying various effects to reproduced 
audio data in response to control inputs from the manual 
controller 16, a parameter changing means 248 for 
changing parameters including an effect level, a sound 

20 volume, etc. of audio data in response to control inputs 
from the manual controller 16, a selecting playback 
means 250 for selecting playback attributes according 
to control inputs, a sound re-sampling means 252 for re- 
sampling audio data of a portion of the reproduced audio 

25 data which is determined by a control input from the 
manual controller 1 6 and registering the re-sampled au- 
dio data in a predetermined playback slot, and a second 
command selecting means 254 for selecting various 
commands after audio data is re-sampled. 

30 [0108] The second command selecting means 254 
comprises the waveform editing means 222 (see FIG. 
9) which removes excessive data from sampled audio 
data assigned to a selected playback slot, and re-sam- 
pled data registering means 256 for registering re-sam- 

35 pied audio data in the audio data file in the sound buffer 
90. 

[0109] The sound re-sampling means 252 uses a 
playback slot file 258 which has four records corre- 
sponding to respective four playback slots, for example, 

40 that are handled by the sound sampling means 252. 
Each of the four records of the playback slot file 258 
stores audio data in a selected recording slot. Particu- 
larly, the fourth record corresponding to the fourth play- 
back slot stores re-sampled audio data. 

45 [0110] A processing sequence of the audio data 
processing means 200 will be described below with ref- 
erence to FIGS. 9, 10, 11 through 23, and through 24 
through 26. 

[0111] In step S1 shown in FIG. 11, the CD checking 
so means 202 displays a message prompting the user to 
load a music CD in the optical disk drive 70 on the dis- 
play monitor 18. 

[01 1 2] In step S2, the CD checking means 202 waits 
for an optical disk 20 to be loaded. If an optical disk 20 
55 is loaded in the optical disk drive 70, then control goes 
to step S3 in which the CD checking means 202 checks 
the loaded optical disk 20. 

[01 13] In step S4, the CD checking means 202 checks 
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if the loaded optical disk 20 is a music CD or not. If the 
loaded optical disk 20 is not a music CD, then control 
goes back to step St , displaying a message prompting 
the user to load a music CD in the optical disk drive 70 
on the display monitor 18. If the loaded optical disk 20 
is a music CD, then control proceeds to step S5 in which 
the sampling means 204 performs its own processing 
sequence. 

[0114] The processing sequence of the sampling 
means 204 will be described below with reference to 
FIGS. 12 and 13. 

[0115] In step S101 shown in FIG. 12, the sampling 
means 204 displays a sampling view 300 (see FIG. 24) 
on the display monitor 18. The sampling view 300 has 
a playback state display area 302 indicative of playback 
states, such as a playback track and a playback time, 
of the music CD, an operation display area 304 for ap- 
plying an effect, changing parameters, and controlling 
the music CD, a waveform display area 306 for display- 
ing a registration destination (recording slot) for sam- 
pled audio data and also displaying the waveform of 
sampled audio data, and a time display area 308 for dis- 
playing a bar representing a sampling time of audio da- 
ta. 

[0116] The operation display area 304 has an effect 
selecting area 310 for selecting various effects, a pa- 
rameter changing area 31 2 for changing various param- 
eters, and a CD control area 314 for controlling a CD. 
[0117] The effects include a type (THRU) for playing 
back audio data without any effect applied, a type 
(ECHO) for applying an echo effect to audio data, a type 
(SPRING) for reverberating audio data using a spring, 
a type (FLANGER) for applying a flanger to audio data, 
etc. The user can select any on of the effects with the 
left button 110d and the right button 110b. 
[0118] The parameters include an effect level (depth: 
EFX), a sound pitch (PIT), and a sound volume (VOL). 
The user can select any on of the parameters with the 
up button 1 1 0a and the down button 1 1 0c, and change 
the numerical value of a selected parameter with the left 
button 1 1 0d and the right button 1 1 0b. 
[0119] The CD control area 314 displays icons for 
playing, stopping, accessing tracks (forward) of, and ac- 
cessing tracks (backward) of , a CD. The user can select 
any one of these icons with the up button 1 1 0a, the down 
button 110c, the left button 110d, and the right button 
110b. 

[0120] In step S102 shown in FIG. 12, the sampling 
means 204 waits for a control input from the manual con- 
troller 1 6. If there is a control input from the manual con- 
troller 16, control goes to step S103 in which the sam- 
pling means 204 determines whether the control input 
is a control input relative to the control of the CD or not. 
If the control input is a control input relative to the control 
of the CD, then control goes to step S104 in which the 
CD controlling means 210 controls the CD according to 
the control input. Specifically, the CD controlling means 
210 plays back the CD, stops the CD, accesses a track 



on the CD in a forward direction, or accesses a track on 
the CD in a backward direction. 

[0121] If the control input is not a control input relative 
to the control of the CD, then control goes to step S1 05 

5 in which the sampling means 204 determines whether 
the control input is a control input relative to sampling, 
e.g., a control input from the start button 40, or not. If 
the control input is a control input relative to sampling, 
then the sound sampling means 216 performs its own 

10 processing sequence. 

[0122] The processing sequence of the sound sam- 
pling means 21 6 will be described below with reference 
to FIG. 14. 

[0123] lnstepS201 shown in FIG. 14, the sound sam- 
is pling means 21 6 determines whether a music CD is be- 
ing played back or not. If a music CD is being played 
back, then control goes to step S202 in which a sampling 
process is prepared. At this time, the color of the dis- 
played frame of the waveform display area 306 changes 
20 to yellow, for example, letting the user know that a sam- 
pling process is being prepared. 

[0124] In step S203, the sound sampling means 216 
waits for a control input from the button 11 2d, serving 
as a decision button, of the manual controller 1 6. If there 
25 is a control input from the decision button 11 2d, then 
control goes to step S204 in which the sound sampling 
means 216 starts sampling audio data that is being re- 
produced. 

[0125] In step S205, the sound sampling means 216 

30 waits for a control input from the decision button 11 2d. 
If there is a control input from the decision button 11 2d, 
then control goes to step S206 in which the sound sam- 
pling means 216 finishes the sampling process. 
[0126] Thereafter, in step S207, the sound sampling 

35 means 216 registers sampled audio data in a record cor- 
responding to a recording slot being selected, of the 
records in the recording slot file 228. 
[0127] After step S207 or if a music CD is not being 
played back in step S201 , the processing sequence of 

40 the sound sampling means 21 6 is finished. 

[01 28] Control then goes back to the routine shown in 
FIG. 12. If the control input is not a control input relative 
to sampling in step S1 05, then control goes to step S1 07 
in which the sampling means 204 determines whether 

45 the control input is a control input relative to an effect or 
not. If the control input is a control input relative to an 
effect, then the effect applying means 212 performs its 
own processing sequence. Specifically, the user selects 
an effect with a control input in step S1 08, and the effect 

50 applying means 212 applies the selected effect to audio 
data being reproduced by the optical disk drive 70 in 
step S109. 

[0129] If the control input is not a control input relative 
to an effect, then control goes to step S110 in which the 
55 sampling means 204 determines whether the contra I in- 
put is a control input relative to the changing of a param- 
eter or not. If the control input is a control input relative 
to the changing of a parameter, then the parameter 
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changing means 214 performs its own processing se- 
quence. Specifically, the user selects a parameter with 
a control input in step S111, and then the parameter 
changing means 214 changes the numerical value of 
the selected parameter in step S112. Thereafter, in step 
S113, audio data is reproduced according to the 
changed parameter. 

[01 30] If the control input is not a control input relative 
to the changing of a parameter in step S 1 1 0, then control 
goes to step S114 shown in FIG. 13 in which the sam- 
pling means 204 determines whether the control input 
is a control input relative to the selection of a slot, e.g., 
a control input from the R2 button 11 6b, or not. 
[0131] If the control input is a control input relative to 
the selection of a slot, then control goes to step S1 1 5 in 
which the slot selecting means 218 performs its own 
processing sequence. 

[01 32] The processing sequence of the slot selecting 
means 218 will be described below with reference to 
FIG. 15. 

[01 33] I n step S301 shown FIG. 1 5, the slot selecting 
means 21 8 displays a slot selection view (window view), 
not shown, on the display monitor 18. 
[0134] In step S302, the user selects a recording slot 
in which to register sampled audio data by pressing the 
up button 1 1 0a, the down button 1 1 0c, and the decision 
button 11 2d of the manual controller 16. Specifically, 
when the user presses the up button 1 1 0a and the down 
button 110c, a cursor is moved, and when the user 
presses the decision button 1 1 2d, a recording slot where 
the cursor is positioned is selected. Then, when the user 
presses the X button 112c, the selected recording slot 
is decided. 

[0135] Instep S303, the slot selecting means 218 de- 
termines whether the selected recording slot is decided 
or not, if t h e sele c ted recording slot is not decided, con- 
trol goes back to step S302 to select a recording slot 
again. If the selected recording slot is decided, then con- 
trol goes to step S304 in which the slot selecting means 
21 8 turns off the slot selection view. The processing se- 
quence of the slot selecting means 21 8 now comes to 
an end. 

[0136] Control then goes back to the routine shown in 
FIG. 13. In step S114, if the control input is not a control 
input relative to the selection of a slot, then control goes 
to step S116 in which the sampling means 204 deter- 
mines whether the control input is a control input relative 
to the selection of a command, e.g. , a control input from 
the selection button 42, or not. 

[0137] If the control input is a control input relative to 
the selection of a command, then control goes to step 
S117 in which the first command selecting means 220 
performs its own processing sequence. 
[01 38] The processing sequence of the first command 
selecting means 220 will be described below with refer- 
ence to FIG. 16. 

[0139] In step S401 shown in FIG. 16, the first com- 
mand selecting means 220 displays a command re- 



quest view, not shown, on the display monitor 18. The 
command request view comprises a window view with 
an array of three commands including a mode changing 
command (MODE CHANGE), a waveform editing com- 

5 mand (WAVE EDIT), and an end command (EXIT). 
[0140] In step S402, the user selects a command by 
pressing the left button 1 1 0d, the right button 1 1 0b, and 
the decision button 11 2d of the manual controller 16. 
[0141] In step S403, the first command selecting 

10 means 220 determines whether the selected command 
is decided or not. If the selected command is not decid- 
ed, control goes back to step S402 to select a command 
again. If the selected command is decided, then control 
goes to step S404 in which the first command selecting 

15 means 220 determines whether the selected command 
is a command for editing a waveform. If the selected 
command is a command for editing a waveform, then 
control goes to step S405 in which the waveform editing 
means 222 performs its own processing sequence. 

20 [0142] The processing sequence of the waveform ed- 
iting means 222 will be described below with reference 
to FIG. 1 7. 

[0143] In step S501 shown in FIG. 17, the waveform 
editing means 222 displays a waveform editing view 320 

25 (see FIG. 25) on the display monitor 18. As shown in 
FIG. 25, the waveform editing view 320 is a window view 
having a waveform display area 322 for displaying the 
waveform of sampled audio data. 
[01 44] In the waveform editing view 320, the user can 

30 trim front and rear portions on a time axis of the dis- 
played waveform. Specifically, the user presses the 
down button 110c to place a cursor 324 in the front or 
rear portion of the displayed waveform, and then press- 
es the left button 1 1 0d orthe right button 1 1 0b to decides 

35 a region (time width) to be trimmed off the displayed 
waveform. After h aving decided the re g ion to b e 
trimmed off the displayed waveform, the user presses 
the decision button 11 2d to trim the region off the dis- 
played waveform. 

40 [0145] In step S502 shown in FIG. 17, the waveform 
editing means 222 displays the waveform of an audio 
data registered in a selected recording slot in the wave- 
form display area 322. 

[0146] In step S503, the user decides a region to be 
« trimmed off the displayed waveform in the manner de- 
scribed above. Thereafter, the waveform editing means 
222 determines whether the region to be trimmed has 
been decided or not in step S504. If not decided, then 
control goes back to step S503 for the user to decide a 
50 region to be trimmed off the displayed waveform again. 
[0147] If decided, then control goes to step S505 in 
which the waveform editing means 222 removes, i.e., 
trims the selected region off the displayed waveform. 
[01 48] In step S506, the waveform editing means 222 
55 reregisters the trimmed audio data in the selected re- 
cording slot. That it, the waveform editing means 222 
re-registers the trimmed audio data in a record corre- 
sponding to the selected recording slot, of the records 
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in the recording slot file. 

[0149] In step S507, the waveform editing means 222 
determines whether the trimming process has been 
completed or not based on whether there is a control 
input from the x button 112c, for example, or not. 
[0150] If the trimming process has not been complet- 
ed, then control goes back to step S503 to select a re- 
gion to be trimmed again. If the trimming process has 
been completed, then control goes to step S508 in which 
the waveform editing means 222 turns off the waveform 
editing view 320. The processing sequence of the wave- 
form editing means 222 is finished. 
[0151] Control then returns to the routine shown in 
FIG. 1 6. If the selected command is a not command for 
editing a waveform in step S404 shown in FIG. 1 6, then 
control goes to step S406 in which the first command 
selecting means 220 determines whether the selected 
command is a command for re-sampling or not. If the 
selected command is a command for re-sampling, then 
control goes to step S407 in which the re-sampling 
means 224 performs its own processing sequence. The 
processing sequence of the re-sampling means 224 will 
be described later on. 

[01 52] If the selected command is not a command for 
re-sampling in step S406, then control goes to step 
S408 in which the sampled data registering means 226 
registers the sampled audio data which has been reg- 
istered in the selected recording slot, in the audio data 
file, e.g., a user audio data file, in the sound buffer 90. 
At this time, the audio data becomes available for use 
as one sound source for the entertainment apparatus 
12. After step S408, the processing sequence of the 
sampling means 204 is ended. 

[0153] Control then returns to the routine shown in 
FIG. 13. If the control input is not a control input relative 
to the selection of a command in step S 1 1 6, then control 
goes to step S1 1 8 in which the sampling means 204 de- 
termines whether the control input is a control input rel- 
ative to the changing of a CD or not based on whether 
the control input is a control input from the R1 button 
116a or not. 

[0154] If the control input is a control input relative to 
the changing of a CD, then control goes back to step S1 
shown in FIG. 11 to display a message prompting the 
user to change a CD in the optical disk drive 70 on the 
display monitor 18 and wait for a CD to be changed. The 
user can leave this waiting state by pressing the x but- 
ton 112c. 

[01 55] If the control input is not a control input relative 
to the changing of a CD in step S118 shown in FIG. 13, 
then control goes to step S1 1 9 in which another process 
depending on the control input is carried out. 
[0156] The processing sequence of the re-sampling 
means 224 will be described below with reference to 
FIGS. 18 and 19. 

[0157] In step S601 shown in FIG. 1 8, the re-sampling 
means 224 displays a re-sampling view 330 (see FIG. 
26) on the display monitor 1 8. The re-sampling view 330 



has a first slot display area 332 for displaying three se- 
lected recording slots, a second slot display area 334 
for displaying the recording slot of re-sampled audio da- 
ta, a playback attribute display area 336 for applying an 

5 effect, changing parameters, and selecting playback at- 
tributes, and a time display area 350 for displaying a bar 
representing a re-sampling time of audio data. 
[0158] The playback attribute display area 336 has an 
effect selecting area 338 for selecting various effects, a 

10 parameter changing area 340 for changing various pa- 
rameters, and a playback attribute selecting area 342 
for selecting playback attributes. 
[0159] The effect selecting area 338 and the param- 
eter changing area 340 are the same as the effect se- 

15 lecting area 31 0 and the parameter changing area 312 
in the sampling view 300 shown in FIG. 24, and will not 
be described below. The playback attribute selecting ar- 
ea 342 allows the user to select a playback direction and 
playback attributes. The user can select playback at- 

20 tributes by pressing the left button 110d and the right 
button 1 1 0b. The playback attributes that can be select- 
ed include a reverse playback mode (REVERSE), a 
skipping playback mode (SKIP), and a scratch playback 
mode (SCRATCH). 

25 [0160] In step S602 shown in FIG. 1 8, the re-sampling 
means 224 waits for a control input from the manual con- 
troller 1 6. If there is a control input from the manual con- 
troller 1 6, control goes to step S603 in which the re-sam- 
pling means 224 determines whether the control input 

30 is a control input relative to the assignment of a slot, e. 
g., a control input from the R2 button 1 1 6b, or not. If the 
control input is a control input relative to the assignment 
of a slot, then control goes to step S604 in which the slot 
assigning means 240 performs its own processing se- 

35 quence. 

[0161] The processing sequence of the slot assigning 
means 240 will be described below with reference to 
FIG. 20. 

[0162] lnstepS701 shown in FIG. 20, the slot assign- 
40 ing means 240 displays a slot assigning view, not 
shown, which comprises a window view having a play- 
back slot display area for displaying four playback slots 
and a recording slot display area for displaying six re- 
cording slots. 

45 [0163] In step S702, the user selects a playback slot 
by pressing the L1 button 114a and the L2 button 114b. 
Then, in step S703, the user selects a recording slot by 
pressing the up button 1 1 0a and the down button 1 1 0c. 
[0164] In step S704, the slot assigning means 240 de- 

50 termines whether the selected slots have been decided 
or not based on whether there is a control input from the 
decision button 11 2d or not. If the selected slots have 
not been decided, then control goes back to step S702 
for the user to select a playback slot and a recording slot 

55 again. 

[0165] If the selected slots have been decided, then 
control goes to step S705 in which the slot assigning 
means 240 assigns the audio data registered in the se- 
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lected recording slot to the selected playback slot. For 
example, the audio data registered in the second re- 
cording slot is assigned to the first playback slot, the au- 
dio data registered in the third recording slot is assigned 
to the second playback slot, the audio data registered 
in the fourth recording slot is assigned to the third play- 
back slot, and the audio data registered in the sixth re- 
cording slot is assigned to the fourth playback slot. 
[01 66] In step 706, the slot assigning means 240 de- 
termines whether the assignment of a slot has been 
completed or not based on whether there is a control 
input from the x button 1t2c or not. If the assignment 
of a slot has not been completed, then control goes back 
to step S702 to assign a slot again. If the assignment of 
a slot has been completed, then control goes to step 
S707 in which the slot assigning means 240 turns off 
the slot assigning view. The processing sequence of the 
slot assigning means 240 now comes to an end. 
[0167] Control then returns to the routine shown in 
FIG. 1 8. If the control input is not a control input relative 
to the assignment of a slot in step S603, then the re- 
sampling means 224 determines whether the control in- 
put is a control input relative to a playback process, i.e., 
a control input from the A button 112a,theDbutton 112b, 
the x button 1 1 2c, or the decision button 11 2d, or not in 
step S605. 

[0168] If the control input is a control input relative to 
a playback process, then control goes to step S606 in 
which the playback processing means 242 performs its 
own processing sequence. 

[0169] The processing sequence of the playback 
processing means 242 will be described below with ref- 
erence to FIG. 21 . 

[0170] In step S801 shown in FIG, 21, the playback 
processing means 242 reproduces audio data in a play- 
back slot depending on the control input. Specifically, if 
the control input comes from the A button 1 1 2a, then the 
playback processing means 242 reproduces audio data 
in the first playback slot. If the control input comes from 
thed button 112b, then the playback processing means 
242 reproduces audio data in the second playback slot. 
If the control input comes from the x button 112c, then 
the playback processing means 242 reproduces audio 
data in the third playback slot. If the control input comes 
from the decision button 11 2d, then the playback 
processing means 242 reproduces audio data in the 
fourth playback slot. 

[0171] In step S802, the playback processing means 
242 determines whether there is a control input indica- 
tive of a playback hold mode, e.g., a control input from 
the R1 button 116a, or not. If there is a control input in- 
dicative of a playback hold mode, then control goes to 
step S803 in which the playback processing means 242 
holds the playback of the audio data assigned to the se- 
lected playback slot. 

[01 72] If there is not a control input indicative of a play- 
back hold mode in step S802, then control goes to step 
S804 in which the playback processing means 242 waits 



for a control input to end. If a control input comes to an 
end, then control goes to step S805 to finish the play- 
back process. 

[0173] After step S803 or step S805, the processing 
s sequence of the playback processing means 242 is 
brought to an end. 

[0174] Then, control returns to the routine shown in 
FIG. 18. If the control input is no a control input relative 
to a playback process in step S605, then control goes 

10 to step S607 in which the re-sampling means 224 de- 
termines whether the control input is a control input rel- 
ative to re-sampling or not. If the control input is a control 
input relative to re-sampling, then control goes to step 
S608 in which the sound re-sampling means 252 per- 

15 forms its own processing sequence. The processing se- 
quence of the sound re-sampling means 252 will be de- 
scribed later on. 

[01 75] If the control input is not a control input relative 
to re-sampling in step S607, then control goes to step 

20 S609 in which the re-sampling means 224 determines 
whether the control input is a control input relative to an 
effect or not. If the control input is a control input relative 
to an effect, then the effect applying means 246 per- 
forms its own processing sequence. Specifically, the us- 

25 er selects an effect with a control input in step S610, and 
the effect applying means 246 applies the selected ef- 
fect to audio data being reproduced by the optical disk 
drive 70 in step S611 . 

[0176] In the step S609, if the control input is not a 

30 control input relative to an effect, then control goes to 
step S61 2 (see FIG. 1 9) in which the re-sampling means 
224 determines whether the control input is a control in- 
put relative to the changing of a parameter or not. If the 
control input is a control input relative to the changing 

35 of a parameter, then the parameter changing means 248 
performs its own processing sequence. Specifically, the 
user selects a parameter with a control input in step 
S613, and then the parameter changing means 248 
changes the numerical value of the selected parameter 

40 in step S61 4. Thereafter, in step S61 5, audio data is re- 
produced according to the changed parameter. 
[01 77] if the control input is not a control input relative 
to the changing of a parameter in step S61 2, then control 
goes to step S616 in which the re-sampling means 224 

45 determines whether the control input is a control input 
relative to the selection of a playback attribute or not. If 
the control input is a control input relative to the selection 
of a playback attribute, then the selecting playback 
means 250 performs its own processing sequence. 

so Specifically, the user selects a playback attribute with a 
control input in step S617, and the selecting playback 
means 250 reproduces audio data according to the se- 
lected playback attribute in step S618. 
[01 78] If the control input is not a control input relative 

55 to the selection of a playback attribute in step S61 6, then 
control goes to step S619 in which the re-sampling 
means 224 determines whether the control input is a 
control input relative to the selection of a slot. A slot can 
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be selected with the L1 button 114a and the L2 button 
114b, for example. 

[01 79] If the control input is a control input relative to 
the selection of a slot, then control goes to step S620 in 
which the slot selecting means 244 selects a playback 
slot to be played back according to a control input. 
[0180] The processing sequence of the sound re- 
sampling means 252 in step S608 will be described be- 
low with reference to FIG. 22. 

[0181] In step S901 shown in FIG. 22, the sound re- 
sampling means 252 determines whether audio data is 
being reproduced or not. If audio data is being repro- 
duced, then control goes to step S902 in which a re- 
sampling process is prepared. At this time, the color of 
the displayed frame of the second slot display area 334 
changes to yellow, for example, letting the user know 
that a re-sampling process is being prepared. 
[0182] In step S903, the sound re-sampling means 
252 waits for a control input from the decision button 
1 1 2d. If there is a control input from the decision button 
1 1 2d, then control goes to step S904 in which the sound 
re-sampling means 252 starts re-sampling audio data 
that is being reproduced. 

[0183] In step S905, the sound re-sampling means 
252 waits for a control input from the decision button 
1 12d. If there is a control input from the decision button 
1 1 2d, then control goes to step S906 in which the sound 
re-sampling means 252 finishes the re-sampling proc- 
ess. 

[0184] Thereafter, in step S907, the sound re-sam- 
pling means 252 registers re-sampled audio data in the 
fourth playback slot, i.e., in a record corresponding to 
the fourth playback slot, of the records in the playback 
slot file 258. 

[0185] After step S907 or if audio data is not being 
reproduced in step S901 , the processing sequence of 
the sound re-sampling means 252 is finished. 
[0186] Control then goes back to the routine shown in 
FIG. 1 9. If the control input is not a control input relative 
to the selection of a slot in step S619, then control goes 
to step S621 in which the sound re-sampling means 252 
determines whether the control input is a control input 
relative to the selection of a command, e.g., a control 
input from the selection button 42, or not. 
[01 87] If the control input is a control input relative to 
the selection of a command, then control goes to step 
S622 in which the second command selecting means 
254 performs its own processing sequence. 
[0188] The processing sequence of the second com- 
mand selecting means 254 will be described below with 
reference to FIG. 23. 

[0189] In step S1001 shown in FIG. 23, the second 
command selecting means 254 displays a command re- 
quest view, not shown, on the display monitor 18. The 
command request view comprises a window view with 
an array of three commands including a mode changing 
command (MODE_CHANGE), a waveform editing com- 
mand (WAVE_EDIT), and an end command (EXIT). 



[01 90] In step S1 002, the user selects a command by 
pressing the left button 1 1 0d, the right button 1 1 0b, and 
the decision button 11 2d of the manual controller 16. 
[0191] In step S1 003, the second command selecting 
5 means 254 determines whether the selected command 
is decided or not. If the selected command is not decid- 
ed, control goes back to step S1002 to select a com- 
mand again. If the selected command is decided, then 
control goes to step S1004 in which the second com- 
10 mand selecting means 254 determines whether the se- 
lected command is a command for editing a waveform. 
If the selected command is a command for editing a 
waveform, then control goes to step S1 005 in which the 
waveform editing means 222 performs its own process- 
's ing sequence. 

[01 92] The processing sequence of the waveform ed- 
iting means 222 is essentially the same as the process- 
ing sequence shown in FIG. 17, and will not be de- 
scribed in detail below. In this processing sequence, the 
20 waveform editing means 222 edits the waveform, i.e., 
trims, the audio data assigned to the selected playback 
slot. 

[01 93] If the selected command is a not command for 
editing a waveform in step S1 004, then control goes to 

25 step S1006 in which the second command selecting 
means 252 determines whether the selected command 
is a command for sampling or not. If the selected com- 
mand is a command for sampling, then control goes to 
step S1007 in which the sampling means 204 performs 

30 its own processing sequence. The processing se- 
quence of the sampling means 204 has been described, 
and hence will not be described below. 
[01 94] If the selected command is not a command for 
sampling in step S1 006, then control goes to step S1 008 

35 in which the re-sampled data registering means 256 reg- 
isters the re-sampled audio data which has been regis- 
tered in the fourth playback slot, in the audio data file, 
e.g., a user audio data file, in the sound buffer 90. At 
this time, the audio data becomes available for use as 

40 one sound source for the entertainment apparatus 12. 
After step S1008, the processing sequence of the re- 
sampling means 224 is ended. 

[0195] Control then returns to the routine shown in 
FIG. 19. If the control input is not a control input relative 

45 to the selection of a command in step S621 , then control 
goes to step S623 in which another process depending 
on the control input is carried out. 
[0196] In the entertainment system 10, extracted au- 
dio data of audio data introduced from an external 

so source is registered as one of sound sources for the en- 
tertainment apparatus 12. Therefore, audio data sup- 
plied from music compact discs (CDs) or via a network 
can be used as a sound source for video games to be 
played back on the entertainment apparatus 1 2, provid- 

55 ing an additional element of interest for video games. 
[0197] Particularly, the waveform editing means 222 
for removing excessive data from the extracted audio 
data can cut off unwanted data from the extracted audio 
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data, leaving only Impressive audio data for use as a 
sound source. 

[0198] Furthermore, since the effect applying means 
212, 246 for applying desired effects to the extracted 
audio data are provided, even one form of audio data 
can be modified into different forms of audio data by ap- 
plying various effects, resulting in a variety of sound 
sources available. 

[01 99] The re-sampling means 224 is provided for re- 
extracting desired audio data from the extracted audio 
data. Consequently, the user can extract only favorite 
portions from the audio data to which various effects 
have been applied orthe audio data from which unwant- 
ed portions have been removed. Therefore, the quality 
of audio data for use as sound sources can be in- 
creased, and the process of extracting audio data is 
easy. 

[0200] Moreover, the selecting playback means 250 
is provided for reproducing the extracted audio data ac- 
cording to one of a plurality of playback attributes which 
is selected by a control input from the manual controller 
16. Therefore, the audio data can be varied in various 
playback modes such as the reverse playback mode 
and the skipping playback mode, resulting in a variety 
of sound sources available. 

[0201] In the illustrated embodiment, audio data is in- 
troduced from a music CD loaded in the optical disk 
drive 70 of the entertainment apparatus 12. However, 
audio data may be introduced from a music playback 
device such as another optical disk drive, a DAT (Digital 
Audio Tape recorder), orthe like connected to an exter- 
nal interface, e.g., a parallel I/O interface, of the enter- 
tainment apparatus 12, with the music playback device 
being controlled by the CPU 72. 
[0202] The audio data processing means can be ex- 
ecuted without having to replace an optical disk which 
stores the audio data processing means and various da- 
ta, with a music CD, so that the entertainment system 
can be operated with ease and the optical disk can be 
replaced with various music CDs quickiy. 
[0203] Although particular embodiments have been 
described herein, it will be appreciated that the invention 
is not limited thereto and that many modifications and 
additions thereto may be made within the scope of the 
invention. For example, various combinations of the fea- 
tures of the following dependent claims can be made 
with the features of the independent claims without de- 
parting from the scope of the present invention. 



Claims 



tainment apparatus (12); 
audio output means (88, 90) for selecting and 
outputting one of a plurality of sound sources 
based on a supplied sound output command; 
5 a display unit (18) for displaying images output- 

ted from said entertainment apparatus (12); 
and 

audio data processing means (200) for regis- 
tering extracted one of audio data introduced 
10 from an external source as one of said sound 

sources for said entertainment apparatus (12). 

2. An entertainment system according to claim 1 , 
wherein said audio data processing means (200) 

15 comprises: 

audio data extracting means (216) for extract- 
ing audio data from the audio data introduced 
from the external source based on a control in- 
20 put entered from said manual controller (16); 

and 

audio data registering means (226) for register- 
ing the extracted audio data as one of said 
sound sources. 

25 

3. An entertainment system according to claim 1 or 2, 
wherein said audio data processing means (200) 
comprises: 

trimming means (222) for trimming an exces- 
30 sive portion off said extracted audio data. 

4. An entertainment system according to any one of 
claims 1 to 3, wherein said audio data processing 
means (200) comprises: 

35 effect applying means (212) for applying an 

effect to said extracted audio data. 

5. An entertainment system according to any one of 
claims 1 to 4, wherein said audio data processing 

40 means (200) comprises: 

audio data re-extracting means (224) for re- 
extracting audio data from said extracted audio da- 
ta. 

45 6. An entertainment system according to claim 5, 
wherein said audio data re-extracting means (224) 
comprises: 

selecting playback means (250) for reproduc- 
ing said extracted audio data according to a play- 
50 back attribute selected according to a control input 
entered from said manual controller (16). 



1 . An entertainment system comprising: 

an entertainment apparatus (12) for executing 55 
various programs; 

at least one manual controller (16) for entering 
control requests from the user into said enter- 



7. An entertainment apparatus for connection to a 
manual controller (1 6) for outputting at least a con- 
trol request from the user, a display unit (18) for dis- 
playing images, and audio output means (88, 90) 
for selecting and outputting one of a plurality of 
sound sources based on a supplied sound output 
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command, comprising: 

audio data processing means (200) for regis- 
tering extracted one of audio data introduced from 
an external source as one of said sound sources. 

8. A recording medium storing a program and data for 
use in an entertainment system (10) having an en- 
tertainment apparatus (12) for executing various 
programs, at least one manual controller (1 6) for en- 
tering control requests from the user into said en- 
tertainment apparatus (12), audio output means 
(88, 90) for selecting and outputting one of a plural- 
ity of sound sources based on a supplied sound out- 
put command, and a display unit (1 8) for displaying 
images outputted from said entertainment appara- 
tus (12), said program comprising the step of: 

registering extracted one of audio data intro- 
duced from an external source as one of said sound 
sources for said entertainment apparatus (12). 

9. A recording medium according to claim 8, wherein 
said step comprises the steps of: 

extracting audio data from the audio data intro- 
duced from the externa! source based on a con- 
trol input entered from said manual controller 
(16); and 

registering the extracted audio data as said one 
of sound sources. 

10. A recording medium according to claim 8 or 9, fur- 
ther comprising the step of: 

trimming an excessive portion off said extract- 
ed audio data. 

1 1 . A recording medium according to any one of claims 
8 to 10, further comprising the step of: 

applying an effect to said extracted audio da- 
ta. 

1 2. A recording medium according to any one of claims 
8 to 11, further comprising the step of: 

re-extracting audio data from said extracted 
audjo data. 

1 3. A recording medium according to claim 1 2, wherein 
said step of re-extracting audio data comprises the 
step of: 

reproducing said extracted audio data ac- 
cording to a playback attribute selected according 
to a control input entered from said manual control- 
ler (16). 



tertainment apparatus (12), audio output means 
(88, 90) for selecting and outputting one of a plural- 
ity of sound sources based on a supplied sound out- 
put command, and a display unit (1 8) for displaying 
5 images outputted from said entertainment appara- 
tus (12), said program comprising the step of: 

registering extracted one of audio data intro- 
duced from an external source as one of said sound 
sources for said entertainment apparatus (12). 

10 
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20 
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30 
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45 



14, A program readable and executable by a computer, 
for use in an entertainment system (10) having an 55 
entertainment apparatus (12) for executing various 
programs, at least one manual controller (1 6) for en- 
tering control requests from the user into said en- 
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